Data Centric AI

Charles F Vardeman II

2023-09-05

How do we develop a curriculum for training large language models?

The “Pile”

LLama: Open and Efficient Foundation Language Models

Touvron, Hugo, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux,

Timothée Lacroix, Baptiste Rozière, et al. “LLaMA: Open and Efficient Foundation Language Models.”

arXiv, February 27, 2023. https://doi.org/10.48550/arXiv.2302.13971.

(GPT-4) “Sparks of AGI”?

Sparks of AGI: Early Experiments with GPT-4, 2023.

https://www.youtube.com/watch?v=qbIk7-JPB2c.

Textbooks Are All You Need!

Textbooks Are All You Need, 2023.

https://www.youtube.com/watch?v=24O1KcIO3FM.

Textbooks Are All You Need!

Gunasekar, Suriya, Yi Zhang, Jyoti Aneja, Caio César Teodoro Mendes,

Allie Del Giorno, Sivakanth Gopi, Mojan Javaheripi, et al.

“Textbooks Are All You Need.” arXiv, June 20, 2023.

http://arxiv.org/abs/2306.11644.